Appln. No. 10/678,551 
Amendment dated June 21, 2007 
Regarding Office Action dated April 11, 2007 
Docket No. 7463-20 (CE11418JUI) 

I. REMARKS/ARGUMENTS 

These remarks are submitted in response to the Office Action of April 1 1 , 2007 
(Office Action). As this response is timely filed within the 3-month shortened statutory 
period, no fee is believed due. 

Applicants have amended independent Claims 1, 2, 6, 11, 12, 16, 20 and 21 to 
emphasize certain aspects of Applicants' invention. The amendments are supported 
throughout the Specification. No new matter has been introduced by virtue of the 
amendments. 

In paragraph 2, page 2 of the Office Action, Claims 1-21 were rejected under 35 
U.S.C. 102(e) as being anticipated by U.S. Patent Application Publication No. 
2003/0200080 to Galanes et al. (hereinafter Galanes). 

Applicant's wish to thank the Examiner for a thorough review of the present 
application and for taking the time to accurately cite relevant sections of the prior art in 
the rejections. Responsive to the Examiner's rejections, Applicant's have amended the 
claims to define further distinguishing features disclosed in the present application. 
Although Applicants are not individually responding to each prior art citation, Applicant's 
have reviewed all the rejections and the corresponding citations brought forth by the 
examiner during formulation of their response. Applicants believe they have adequately 
amended the claims to further clarify the distinguishing features of the invention not 
taught or contemplated by the prior art. 

II. Applicant's Invention 

It may be helpful to reiterate certain aspects of Applicant's invention prior to 
addressing the references cited in the Office Action. One embodiment, for example, 
provides in a device, a method for unifying speech user interface and graphic user 
interface commands, comprising the steps of receiving grammar specifying a syntax of 
at least one speech command and having semantic information, processing the 
grammar to extract the semantic information for use with both a graphical user interface 
and a speech user interface, receiving a user input to select at least one component in 
the graphical user interface during a navigation of the graphical user interface, audibly 
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receiving the at least one speech command associated with the at least one component 
from the speech user interface, visually presenting the at least one speech command in 
a visual prompt of the graphical user interface responsive to the step of audibly 
receiving, and building a corresponding speech command for a plurality of speech 
commands associated with a plurality of components. The corresponding speech 
command is a single instance of a plurality of speech commands associated with a 
selection of one or more components of the graphical user interface during the 
navigation of the graphical user interface. 

III. The Claims Define Over the Prior Art 

As already noted, independent Claims 1, 2, 6, 11, 12, 20, 21 were rejected under 
35 U.S.C. 102(e) as being anticipated by Galanes. Applicant respectfully submit, 
however, that Galanes does not teach or suggest each feature recited in amended 
Claims 1,2, 6, 1 1 , 12, 16, 20 and 21 . 

Galanes is directed to controls for a web server that generates client side markup 
enabled with recognition and/or audible prompting. Controls commonly related to visual 
rendering are extended to include attributes related to recognition and/or audible 
prompting. Typically, controls such as a "label" use a library having markup information, 
which provides a visual prompt on a display. Similarly, a "textbox" provides an input field 
on a visual display. In a first embodiment, an additional library is provided for 
recognition and/or audibly prompting, wherein the controls include attributes or 
parameters to use both libraries (See Galanes paragraph [0010]). 

Galanes also discloses an embodiment wherein a set of companion controls 
having attributes related to recognition and/or audible prompting are formed. The 
companion controls use a library having recognition and audibly prompting markup 
information. The companion controls are selectively associated with visual controls. In 
this manner, application logic can remain with the visual controls, wherein the 
companion controls provide recognized results to the visual controls. The companion 
controls follow a dialog in that controls are provided for prompting a question, obtaining 
an answer, confirming a result, providing a command, or making a statement (See 
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paragraph [0012]). 

Although Galenes discloses extending recognition and audible prompting in a 
graphical user interface, and a set of companion controls for providing client-side mark- 
up information, Galanes does not teach receiving a user input to select at least one 
component associated with a label in the graphical user interface during a navigation of 
the graphical user interface, audibly receiving the at least one speech command 
associated with the at least one component from the speech user interface, visually 
presenting the at least one speech command in a visual prompt of the graphical user 
interface responsive to the step of audibly receiving, and building a corresponding 
speech command for a plurality of speech commands associated with a plurality of 
components. 

More specficially, Galenes does not teach a system that during a navigation of a 
graphical user interface (GUI) identifies user inputs (e.g. text input, mouse click) 
associated with a selection of components (e.g. buttons, lists, links) in the GUI, audibly 
receives a speech command (e.g. spoken utterance) associated with a selected 
component, associates the speech command with the components, or a label (e.g. 
"name") of the selected component, visually presents the speech command, for 
example, in text format, in a visual prompt of the GUI during the navigation of the GUI, 
and builds a corresponding speech command that is a single instance of a plurality of 
speech commands received in association with a selection of components during the 
navigation of the GUI. 

For example, as shown in FIG. 3 of Applicant's drawings, a user can audibly say 
speech commands (e.g. spoken utterance) during a navigation of the GUI 300 while 
selecting GUI components. For instance, the user can select "switch to" on a menu bar 
of the GUI (subplot 1) using a mouse click or keystroke command, and then say "Switch 
to Lookup". The GUI captures the audible spoken command then switches to the 
Lookup dialogue (subplot 2) responsive to a selection of the Lookup component, and 
associates the spoken utterance "Switch to Lookup" with the plurality of (i.e. 
combination of) of selected components (i.e. "Switch too" + "Lookup"). In such regard a 
single instance for a spoken command can be created and visually presented on the 
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GUI. Similarly, when the user audibly says "Lookup entry with" and selects the name 
component in the dialogue, the GUI associates the Lookup entry dialog with the "name" 
selection. The user (subplot 3) can then say "Lookup entry with name Mary Smith" 
responsive to selecting Mary in the dialogue. In such regard, the user builds a 
corresponding speech command during the navigation of the GUI. 

Upon building the corresponding speech command as shown in FIG. 3, the next 
time the user accesses the GUI, as shown in FIG. 4, the visual prompt 402 visually 
presents the spoken commands previously captured and associated with previously 
selected components during audible navigation of the GUI, thereby allowing the user to 
remember the phrases the speech user interface (SUI) associated with the component 
selections. Moreover, the user during the audible presentation establishes the speaking 
style. For example, instead of saying "switch to" followed by a pause to allow the SUI to 
capture the single voice tag, then saying "name" followed by another pause, the user 
can speak a phrase they deem suitable for the immediate application. For instance, the 
user may want to say the label "lookup entry with" instead of "lookup'V'name". 

In such regard, the user is reminded of the phrases that allow the user to 
traverse menu selection of components in a GUI. For example, as the user speaks, the 
GUI automatically navigates through selected components responsive to the SUI 
recognizing corresponding spoken commands (See paragraph [0017] "...the GUI can 
also include a window 402 where the speech interface command is displayed as it is 
being constructed by the user using the GUI, further enhancing the user's memory of 
the speech commands".) Also as noted in paragraph [0004] of the Specification, "This 
makes it easier for the user to give the correct commands to operate a device. It also 
provides a mechanism to display the corresponding speech command when the user 
uses the GUI to give the command. This further reinforces the speech command in the 
user's memory, reducing the errors in giving the command by speech". That is, the user 
can visually see the building of the corresponding speech command. 

Notably, the GUI menus and dialog box elements are constructed in such a way 
that by selecting them (e.g. text entry, keystroke selection, mouse click) during 
navigation, the user essentially builds the corresponding speech command for an 
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application (See specification paragraph [0003] describing the building of a 
corresponding speech command). The building of a corresponding speech command 
that is visually presented during a navigation of a GUI is a feature that is not taught or 
contemplated in the prior art. 

With regard to claim 10, on page 5 of the office action, it was stated that Galanes 
teaches forming a window for displaying a speech interface command as it is being built 
using the graphical interface (see Galanes, paragraph [0043] lines 7-9). Applicant's 
respectfully disagree. At the cited section, and throughout, Galanes only contemplates 
forming controls having attributes related to recognition and/or audible prompting. 
Galanes does not generate a visual prompt that identifies the spoken command spoken 
by the user during navigation of a GUI. Although, Galanes discloses that some of the 
components of the GUI include a "label" provide a visual prompt on a display, or a 
"textbox" to provide an input field on a visual display (see paragraph [0010]), this is not 
equivalent to actively presenting at least one speech command in a visual prompt of the 
graphical user interface responsive receiving the at least one speech command during a 
navigation of the GUI, and building a corresponding speech command for the at least 
one speech command associated with the label. As shown in FIGs. 3 and 4 of 
Applicant's drawings, the user can literally create phrases (e.g. build a corresponding 
speech command by concatenating spoken utterances received while navigating the 
GUI) associated with the navigation of the GUI that are stored and which later allow the 
user to recall the phrases during a subsequent navigation of the GUI. 

Although Galanes discloses forming associations between speech commands 
and graphical user interface components, some of which may have labels, Galanes 
does not contemplate allowing a user to say a phrase (e.g. at least one, or, a plurality of 
spoken commands) that the user associates with the labels or components For 
example, as shown in FIG. 4, the user can associate the phrase "Lookup entry with the 
name Mary Smith" with the corresponding sequence of GUI component selections 
SwitchTo-Lookup-name-MarySmith to navigate the user through the GUI to Mary 
Smith's information page. As shown in FIG. 4, the visual prompt, which presents the 
spoken commands available to the user while the user navigates the GUI, produces 
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redundant textual information. For instance, the text "Lookup entry with name" in the 
voice prompt 402 is also seen in the label of the text list, and the "name" entry in the text 
list. In such regard, the voice prompt presents repetitive information. 

The presentation of the spoken utterance in a visual prompt is one novel aspect 
of Applicant's invention. As stated previously, the voice prompt allows a user to see the 
spoken commands, which helps the user in remembering what phrases are available to 
the user to navigate the GUI. In such regard, the redundant aspect of providing the 
visual prompt reassures the user that the commands they are speaking correspond to 
components in the GUI. Galanes does not contemplate a visual prompt that presents 
possibly redundant text corresponding to one or more components or labels in a GUI. 
Moreover, Galanes does not teach or contemplate displaying a speech interface 
command as it is being built using the graphical user interface as expressed in claim 10. 

With regard to claim 6, on page 4 of the Office Action it was also stated that 
Galanes teaches the steps of receiving user entered text via a graphical user interface, 
and updating a speech recognizer with the user entered text and semantic information. 
However, Galanes does not teach monitoring the user entered text and adding input 
context associated with a selection of at least one component in the graphical user 
interface to the user entered text as recited in amended claim 6. Input context is the 
selection of components in the GUI during a navigation of the GUI. For example, 
referring to FIG. 4, the input context is the sequence of selections made by the user 
when the user selects components for navigating the SwitchTo-Lookup-name- 
MarySmith components. Moreover, Galanes does not teach updating a speech 
recognizer with the user entered text and semantic information, audibly receiving at 
least one speech command associated with the at least one component from the 
speech user interface, visually presenting the at least one speech command in a visual 
prompt of the graphical user interface responsive to the step of audibly receiving, and 
building a corresponding speech command for a plurality of speech commands 
associated with a plurality of components, wherein the corresponding speech command 
is a single instance of a plurality of speech commands associated with a selection of 



14 



Appln. No. 10/678,551 
Amendment dated June 21, 2007 
Regarding Office Action dated April 11, 2007 
Docket No. 7463-20 (CE11418JUI) 

one or more components of the graphical user interface during the navigation of the 
graphical user interface. 

IV. CONCLUSION 

Applicant's have shown that the reference fails to teach or suggest every feature 
recited in amended independent Claims 1 , 2, 6, 11, 12, 16, 20 and 21. Applicants 
respectfully submit, therefore, that each of Claims 1 , 2, 6, 1 1, 12, 16, 20 and 21 defines 
over the prior art. Applicants further respectfully submit that whereas each of the 
remaining claims depends from one of the amended independent claims while reciting 
additional features, these dependent claims likewise define over the prior art 

Applicants believe that this application is now in full condition for allowance. 
Allowance is therefore respectfully requested. Applicants request that the Examiner call 
the undersigned if clarification is needed on any matter within this Amendment, or if the 
Examiner believes a telephone interview would expedite the prosecution of the subject 
application to completion. 

Respectfully submitted, 



Date: June 22, 2007 /Pablo Meles/ 

Pablo Meles, Reg. No. 33,739 

Marc A. Boillot, Reg. No. 56,164 
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Customer No. 55794 

Post Office Box 3188 

West Palm Beach, FL 33402-3188 
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